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Exploring large panoramas for Mars 


MER Image Browser was developed at Ames and was used in 
MER operations for interactively exploring panoramas 








MER Landing Full panorama 
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MER Image Browser 


Address/filenames: |browse:merb&images&sol=2 Go | Back | 

Browse: C Mer A <? Mer B 

Collate by: (* Sol only C 3ase motion fz i] 

Projection: (? Cylindrical projection C Spherical projection C Unprojected C 
Stereo pairs: <* L+R individually C L only C R only C C 

Instruments: [7 Front hazcams [7 Rear hazcams [7 Navcams |7 Pancams [7 Ml 
Draw: C Images (♦* Images with borders C Borders only 
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Zooming to 6X shows rock outcrop 
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MER Image Browser 




Go 


^ 

Address/filenames: |browse:merb&images&sol=2 

Browse: C Mer A <? Mer B 

Collate by: (? Sol only C 3ase motion [2 |] 

Projection: (? Cylindrical projection C Spherical projection C Unprojected C 
Stereo pairs: <? L+R individually C L only C R only C C 

Instruments: [7 Front hazcams [7 Rear hazcams [7 Navcams [7 Pancams [7 Ml 
Draw: C Images G Images with borders C Borders only 
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Zooming to 14X reveals layers 
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MER Image Browser 



ddress/filenames: |brow$e:merb&images&$ol=2 Go 

Irowse r Mer A (• Mer B 
ollate by & Sol only C |i |] 

‘rejection G Cylindrical projection C Spherical projection C Unprojectec 
Stereo pairs G L*R individually C L only C R only C 
Instruments; F Front hazcams F Rear harcams F Navcams F Pancams 
Draw C Images G Images with borders C Borders only 
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Zooming to 40X: We’ve found bedrock! 
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A (short) History of the Gigapan software 


• In 2004, Randy Sargent, Larry Edwards, Matt Deans, and Anne Wright 
are working at NASA Ames Research Center in the robotics group run 
by lllah Nourbakhsh 

• They developed methods for integrating multiple images from the Mars 
Exploration Rover quickly 

• Randy became infatuated with the panoramas from Mars, both 
explorable on computers and printed on big banners on the wall 

• Randy used some of the software Ames developed to mosaic images 
on the fly and to zoom around to explore; these later became the basis 
for the GigaPan stitcher and viewer 

• (caveat: JPL who also produced mosaics that might not have been as explorable, 
but were the "official" products) 


• lllah and Randy talked whether they could make something very low- 
cost that would do the same thing here on Earth. => Global Connection 
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Gigapixel Panoramas 


With consumer-grade digital cameras, 
Gigapixel panoramas are cheap and 
easy to capture 


This prototype device developed by 
Randy’s team began capturing half- 
gigapixel panoramas in a backyard 
in California every half-hour starting 
in August 2004 
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Panorama of yard 
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Global Connection (GC) 


Goals 

• Use spatial images to 
connect, inform, and inspire 

• Create explorable imagery 

• Encourage global citizenship 
and understanding 

GigaPan 



- Gouglr 


ania market 
yaflyover” project) 


• Gigapixel panoramas 



• Low-cost (~$300) robotic 
pan/tilt head + stitching 
software 

• Education, exploration & 
science 

• Community, culture & 
journalism 

Partners CMU, Google, 
Nat’l Geographic, NASA 



gigapan.org 
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NASA Exploration: 

What’s Changed Since Apollo? 










Planned Joint NASA-ESA Mars Program 


NASA-ESA Joint Mars Initiative 


Operational 

2001-2007 

Odyssey -i 


2009 


2011 


Mars Express 
Collaboration 


2013 2016 2018 2020 & Beyond 


MAVEN 

Aeronomy 

Orbiter 


ESA— NASA 
ExoMars 
Trace 

Gas Orbiter 


The Era of 
Mars Sample 
Return 
begins 


PJroenix^ 

9HSB! 


Mars jSsience i 
Laboratory 


ESA— EDL 
Demonstrator 


NASA— ESA 
Rovers 


C ft (Astroblology/ 
Sample Return 
Cache) 


"For P anning and Discussion Purposes Only" 



Mars Science Lab comparison with MER 





MSL’s Rover Configuration 



DAN 


MastCam 


REMS 


◄ HGA 


SA/SPaH: 


Arm 


Brush/Abrader 

Corer 


Scoop 

Rock Crusher 


APXS 

MAHLI 


SAM 

CheMin 


MARDI 


ChemCam 


Conceptual Design 


UHF 


Wheel Base: 

1.5 

m 

Height of Deck: 

1.1 

m 

Height of Mast: 

2.1 

m 

Wheel Diameter: 

0.5 

m 

Clearance: 

0.66 m 
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MSL’s Mast Camera (MastCam) 


Principal Investigator: Michael Malin 
Malin Space Science Systems 



MastCam observes the geological structures and 

features within the vicinity of the rover 

• Studies of landscape, rocks, fines, frost/ice, and 
atmospheric features 

• Stereo, zoom/telephoto lens: 15X, from 90° to 6.5° 
FOV 

• Bayer pattern filter design for natural color plus 
narrow-band filters for scientific color 

• High spatial resolution: 1200x1200 pixels (0.2 mm/ 
pixel at 2 m, 10 cm/pixel at 1 km) 

• High-definition video at 5 FPS, 1280x720 pixels 

• Large internal storage: 256 MByte SRAM, 8 GByte 
flash 
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Comparison of Asteroids (to scale) 


~33 km 


~500 km 


~1000 km 

< > 


(433) Eros 



(4) Vesta 


(1) Ceres 


NASA 
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(25143) Itokawa 



Asteroid Itokawa, ISS and Orion 






Hayabusa Touchdown Site Candidate A: 
Muses Sea - JAXA Mission 



Largest smooth 
terrain located 
between the 
“Head” and 
“Body” of the 
Otter-like [shape 
of Itokawa] 

~60 m across at 
its widest point. 

NASA 

* 













Gigapixel Planetary Mapping 


Purpose 

• High-quality digital maps 

• On-line access 

• Very rapid updates 

Data processing 

• Orbital imagers (Moon, Mars, etc.) 

• Image base maps 

• 3D terrain reconstruction (DEM’s) 

Data fusion & retrieval 

• OGC standards (WMS, WFS) 

• Geobrowser markup (KML, WTML) 

• Image metadata (GeoTIFF, etc.) 
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Planetary Data Firehose 
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Source: B. Archinal, L. Gaddis, et al. (2007) 

“Urgent Processing and Geodetic Control of Lunar Data’’ 

Workshop on Science Associated with the Lunar Exploration Architecture 
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Planetary Data Firehose 


Traditional mapping 

• Human-intensive 
cartography 

• Manual control & 
error analysis 

• Maps take years 
to complete 

Image resolution 

• Imagers keep 
getting better 

• High-res digital 
scans of old film 


Apollo Metric Camera 
16,000 x 16,000 



MER Pancam □ 
1 024 x 1024 


Not shown: 
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HiRISE 

20,000x40,000 


LROC 

10000x50000 


Apollo Panoramic Camera 
25400 x 244000 




Automated Stereo Image Processing 
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Automated Stereo Image Processing 


Problem 

• Multiple images from 
different viewing angles 

• Compute 3D terrain model 
with no human intervention 



Open-source C++ packages 

■ NASA Ames Stereo Pipeline 

■ NASA NeoGeography Toolkit 

■ NASA Vision Workbench 




M. Broxton and L. Edwards (LPSC 2008) 
“The Ames Stereo Pipeline: Automated 3D surface reconstruction from orbital imagery” 
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Mars Terrain Models 



MGS MOC Narrow Angle 

• Collaboration with Malin Space 
Science Systems 

• Adapted Ames Stereo Pipeline to 
orbital images 


MRO Context Imager (CTX) 

• Collaboration with CTX Team 

• Provided rapid turn-around 
stereo modeling 



Mars Express HRSC 

• Collaboration with USGS, DLR 

• Formal comparison of Digital 
Elevation Model (DEM) products 

• Four controlled data sets 
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Mars Terrain Models 
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Lunar Terrain Models 



Apollo Panoramic Camera 

• 3D terrain model of 
Taurus-Littrow valley 
(Apollo 17) 

• Featured at the Hayden 
Planetarium (American 
Museum of Nat. History) 


Apollo Metric Camera 

• Systematic creation of 
image maps & DEMs 

• Refinement of the Lunar 
geodetic control network 
(with USGS) 

• NASA Lunar Mapping & 
Modeling Project 







dr 


2 ^ 


Hadley Rille 
(40 m/post DEM +10 m/pixel image) 


3e) 
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Lunar Terrain Models 


Apollo Image Archive project 

• apollo.sese.asu.edu 

(Mark Robinson, ASU) 

• Photogrammetric scans of 
original Apollo films 

(200 pixel/mm, 14-bit) 


A 
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Lunar Terrain Models 


Large-scale mapping 

• Systematic reconstruction of the “Apollo Zone” 

• Stereo vision & mosaicking of Apollo Metric Camera 

• 8,000 stereo pairs from Apollo 15-17 






Apollo 15 Orbit 33 
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M. Broxton, A. Nefian, et al. (ISVC 2009) 

”3D Lunar terrain reconstruction from Apollo Images” 
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Coming Soon ... 


Complete Mars HiRISE Mosaic 

• Mars Reconaissance Orbiter HiRISE imager 

• Each image: 20,000 x 50,000 pixels 

Mosaic stats 


Tile Dimensions 

256 x 256 pixels 

Root Tiles / Image 

15,000 

Tile Space 

25 KB 

Tiles Total 

229 million 

Total Mosaic Size 

5.7 TB 
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NASA’s Participatory Exploration 


Purpose 

• Enable everyone to actively 
participate in NASA missions 

• Explore space in bold, new ways 

• Engage & educate students 

NASA data for everyone 

• Easy access to planetary data 

• Reach millions of users 
(very low barrier to entry) 

• Neo-geography browsers 

(Google Earth, World WideTelescope) 


Citizen science 

• Volunteers help perform science 

• Informal & formal education 

• Social networking 
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NASA Data for Everyone 


Our goals 

• Make NASA’s geospatial data 
universally and easily accessible 

• Enable millions of people to find 
and use NASA data 

• Improve planetary science & 
exploration missions 

How do we do this? 

• Process raw data into maps 
(mosaics, terrain models, etc) 

• Support geo-browsers & GIS 
platforms through open standards 

• Provide incremental updates, so that 
data can be shared in near real-time 
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Mars in Google Earth 




a.k.a. “Google Mars 3D” 

• Launched Feb. 2, 2009 

• Co-developed with Google 

• Built in to Google Earth v5 

Content 

• Global maps: topography, 
infrared, historical, etc. 

• Imager footprints & overlays: 
HiRISE, CTX, MOC, etc. 

• MER tracks & panoramas 

• Tours (Bill Nye & Ira Flatow) 

• Live from Mars: THEMIS 
images within hours 

• And much more ... 


R. Beyer, M. Broxton, et al. (AGU 2009) 
’’Visualizing Mars data and imagery with Google Earth” 
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Moon in Google Earth 




a.k.a. “Google Moon” 

• Launched July 20, 2009 

• Co-developed with Google 

• Built in to Google Earth v5 

Content 

• Global maps: topography, 
geologic, historical, etc. 

• Spacecraft imagery: Apollo, 
Clementine, Lunar Orbiter 

• 3D models of spacecraft, 
landers, and crew rovers. 

• Tours (Andy Chaikin, Buzz 
Aldrin & Jack Schmidt) 

• And much more ... 


M. Weiss-Malik, T. Scharff, et al. (AGU 2009) 
’’Visualizing Moon data and imagery with Google Earth 
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Lunar Analog Site (2009) 




Black Point Lava Flow 

• 65 km N of Flagstaff, AZ 

• Analog of the “Straight Wall” 
(Mare Nubrium / Rupes Recta) 

• Basaltic volcanic rocks & 
unit contacts 
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Surface Robotic Recon Instruments 



3D scanning LIDAR 


• 3D topography 
measurements 

• 5mm @ 500m 

• Oblique views not 
possible from orbit 

Gigapixel Exploration of Space :: 



GigaPan 


• Oblique, wide-angle, 
color, context views 

• 60x180 deg 

• >100x resolution of 
orbital images 

:: David.Korsmeyer@nasa.gov 



Microscopic Imager 

• High-res, close-up, 
color, terrain views 

• 33 micron / pixel 

• >7,000x resolution of 
orbital images 

42 







Orbital Data 


100% scale 
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Digital Globe QuickBird (60 cm/pixel) 
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Surface Data 



The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the 


Je agaij^li^the red x still apoears, you may have to delete the image and then insert it again. 

)Cr;o scale 





GigaPan panorama close-up 


Terrain image (55 microns / pixel) 
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Surface Data 



3D scanning LIDAR (250 m range, 3 mm depth resolution) 
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Collected Recon Data 
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Collected Recon Data 




8.5 GB data collected (52 hrs of robotic recon operations) 

39 LIDAR scans, 75 GigaPan, and 95 terrain images 
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GigaPan on the Lunar Electric Rover 


Desert RATS 2009 

• GigaPan “Voyage” (rugged pan-tilt + embedded processing/server) 

• Remotely operated by science backroom (ground control) 

• Context & high-resolution imaging for field geology 




Traverse N1, Station 1 (Aug 29, 2009) 
290x40 deg panorama (112 images) 
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Participatory Exploration for D-RATS 2010 




Objectives 

• Citizen science using GigaPan 

• Involve public & students 

Pre-test (Dec 2009 - May 2010) 

• Manually take GigaPans on 
preliminary LER traverse plan 

• Public explores panoramas, takes 
snapshots & adds comments 

• On-line discussion w/ scientists 
(places to stop, tasks, etc.) 

• Public input for final plan 

Field-test (Aug - Sept 2010) 

• Take GigaPan panoramas from 
LER & publish via gigapan.org 

• Public explores panoramas & 
discusses discoveries w/ scientists 
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NASA’s Exploration of the Solar System 


• Generates some of the most diverse 
(heterogeneous), data rich (multi-scale, multi-type). 

• Gathers Micro-scale data in a Macro-scale Context 

• Largely (multi-spectral) image-based assessments 

• Relative (location) context of the imagery gained is 
critical data 


Gigapixel-scale Exploration is here to stay 
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